The present invention relates to a method and apparatus for three-dimensional face recognition and, more particularly, but not exclusively to such a method and apparatus that both obtains a 3D representation of a face and uses that representation for matching purposes.
Face recognition has recently become an important task of computer vision, and is needed in a wide range of biometric and security applications. However, most existing face recognition systems are reported to be sensitive to image acquisition conditions such as head position, illumination, etc. and can therefore be inaccurate and easily fooled. Reference is made to American Civil Liberties Union (ACLU) of Florida, Press Release, 14 May 2002, Available: http://www.aclufl.org/pbfaceitresults051402.html.
In general, modern face recognition approaches can be divided into two wide categories: 2D approaches, using only image information (which can be either grayscale or color), and 3D approaches, which incorporate three-dimensional information as well.
While simpler in data acquisition (which permits real-time surveillance applications, such as face recognition from a video-taped crowd in pubic places), the 2D approach suffers from sensitivity to illumination conditions and head rotation. Since the image represents the light reflected from the facial surface at a single observation angle, different illumination conditions can result in different images, which are likely to be recognized as different subjects (see FIG. 3). One of the classical 2D face recognition algorithms is the Turk and Pentland eigenfaces algorithm. For a full discussion see M. Turk and A. Pentland, Face recognition using eigenfaces, CVPR, May 1991, pp. 586-591, and M. Turk and A. Pentland, “Face recognition system, U.S. Pat. No. 5,164,992, 17 Nov. 1990. The Eigenfaces algorithm works as follows: Given a set of faces arising from some statistical distribution, the principal components of this distribution form a set of features that characterize the variation between faces. “Eigenfaces” are the eigenvectors of the set of “all” faces. The eigenface approach treats face recognition as a 2D classification problem, without taking into consideration the significant difference in the images resulting from illumination conditions and head rotation. For this reason, eigenfaces usually produce mediocre results when faced with real life rather than laboratory conditions.
The 3D approach provides face geometry information, and face geometry information is independent of viewpoint and lighting conditions. Thus such information is complementary to the 2D image. 3D information, however, not only carries the actual facial geometry, but includes depth information which allows easy segmentation of the face from the background.
Gordon showed that combining frontal and profile views can improve the recognition accuracy and reference is made to G. Gordon, “Face recognition from frontal and profile views”, Proc. of the International Workshop on Face and Gesture Recognition, Zurich, Switzerland, pp 47-52, June 1995.
Beumier and Acheroy show the adequacy of using geometric information in the rigid face profile for subject identification in a system using structured light for 3D acquisition, and reference is made to C. Beumier and M. P. Acheroy, Automatic Face Identification, Applications of Digital Image Processing XVIII, SPIE, vol. 2564, pp 311-323, July 1995.
The above described approach may be generalized to the whole surface, and reference is made to C. Beumier and M. P. Acheroy, Automatic Face Authentication from 3D Surface. British Machine Vision Conference BMVC 98, University of Southampton UK, 14-17 Sep. 1998, pp 449-458”, 1998, who describe such a generalization using global surface matching. However, surface matching is sensitive to facial expressions and cannot be considered a comprehensive solution.
There is thus a widely recognized need for, and it would be highly advantageous to have, a facial recognition system that uses three-dimensional information but is devoid of the above limitations such as being sensitive to facial expressions, lighting of the subject, or to angle.